# Continued Pretraining

A Japanese-enhanced large language model based on the Gemma-2 architecture, significantly improving Japanese capabilities while retaining original English proficiency

This is a Tinyllama model specifically optimized for Traditional Chinese through continued pretraining, based on the TinyLlama-1.1B architecture with a pretraining dataset of approximately 2 billion tokens.

A Japanese-optimized model based on Meta-Llama-3-8B, continuously pretrained on a mixed dataset of Japanese and English with 22 billion tokens

A large instruction-based language model customized for the legal domain, obtained through continued pretraining on Mistral-7B

Swallow-MS-7b-v0.1 is a Japanese-enhanced model based on Mistral-7B-v0.1 with continued pretraining, developed by TokyoTech-LLM, demonstrating excellent performance on Japanese tasks.